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WHAT IS CLAIMED IS: 
1. An image processing apparatus for assuming a 
characteristic of a virtual image sensing optical 
system, and applying a blur effect corresponding to a 
preset in-focus state to a captured image, comprising: 

an image input unit for capturing image infor- 
mation including distance information to each portion 
of an object to be photographed; 

^a-jlg^r^inp^t unit for inputting Kj?* 1 ^*** ^ 
from which an ^fe^tlje aperture and f^cal length of 
the assumed image sensing optical system can- be 
derived; 

an in-focal pint position designation unit for 
designating an in-focal pinrp^sTtion of the assumed 
image sensing optical system; 

a blur state calculation unit for calculating a 
blur state from the distance information input by said 
image input unit, the in-focal pint position designated 
by said in-focal pint position designation unit, and 
the parameter input by said parameter input unit; and 

an image processing unit for applying the blur 
effect to the image input by said image input unit in 
correspondence with the blur state calculated by said 
blur state calculation unit. 

2. An image processing apparatus for applying a 
blur effect to a captured image, comprising: 

an image input unit for capturing image 
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information including distance information to each 
portion of an object to be photographed; and 

an image processing unit for applying the blur 
effect to the image input by said image input unit by 
5 overwrite sequentially from an image portion having far 

distance information. 

3. An image processing apparatus for applying a 
blur effect to a captured image, comprising: 
an image input unit for capturing image 

10 information; 

an image processing unit having a first operation 
mode of applying the blur effect to only part of the 
image input by said image input unit, and a second 
operation mode of applying the blur effect to a 

15 remaining image p ortion; 

a switching unit ^able^7>xternally switching 
an operation mode of said image processing unit from 
the first operation mode to the second operation mode; 
and 

20 a display unit < tfafable~^ > displaying an image 

blurred by said image processing unit. 

4. An image processing apparatus for assuming a 
characteristic of a virtual image sensing optical 
system, and applying a blur effect corresponding to an 
25 in- focus state to a captured image, comprising: 

an image input unit for capturing image 
information including distance information to each 
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portion of an object to be photographed; 

a parameter input unit for inputting a parameter 
from which an effective aperture and focal length of 
the assumed image sensing optical system can be 
derived; 

an in-focal pint position designation unit for 
designating an in-focal pint position of the assumed 
image sensing optical system; 

a confusion circle calculation unit for 
calculating a confusion circle from the distance 
information input by said image input unit, the 
in-focal pint position designated by said in-focal pint 
position designation unit, and the parameter input by 
said parameter input unit; 
15 a blur state calculation unit for calculating a 

blur state using a point spread function (PSF) within a 
range corresponding to a size of the confusion circle 
calculated by said confusion circle calculation unit; 
and 

an image processing unit for applying the blur 
effect to the image input by said image input unit in 
correspondence with the blur state calculated by said 
blur state calculation unit. 

5. An image processing method of assuming a 
25 characteristic of a virtual image sensing optical 

system, and applying a blur effect corresponding to a 
preset in-focus state to a captured image, comprising 
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the steps of : 

capturing image information including distance 
information to each portion of an object to be 
photographed ; 

inputting a parameter capable of deriving an 
effective aperture and focal length of the assumed 
image sensing optical system; 

designating an in-focal pint position of the 
assumed image sensing optical system; 

calculating a blur state from the input distance 
information, the designated in-focal pint position, and 
the input parameter; and 

applying the blur effect to the input image in 
correspondence with the calculated blur state. 

6. An image processing method of assuming a 
characteristic of a virtual image sensing optical 
system, and applying a blur effect corresponding to an 
in-focus state to a captured image, comprising the 
steps of : 

capturing image information including distance 
information to each portion of an object to be 

photographed ; 

inputting a parameter from which an effective 
aperture and focal length of the assumed image sensing 
optical system can be derived; 

designating an in-focal pint position of the 
assumed image sensing optical system; 



calculating a confusion circle from the input 
distance inf ormation, the designated in-focal pint 
position, and the input parameter; 

calculating a blur state using a point spread 
function (PSF) within a range corresponding to a size 
of the calculated confusion circle; and 

applying the blur effect to the input image in 
correspondence with the calculated blur state. 

7. An image processing method of applying a blur 
effect to a captured image, comprising the steps of: 

capturing image information including distance 
information to each portion of an object to be 
photographed; and 

applying the blur effect to the input image by 
overwrite sequentially from an image portion having far 
distance information. 

8. An article of manufacture comprising: 
a computer-readable storage medium having 

computer-readable program code means stored to assume a 
characteristic of a virtual image sensing optical 
system and apply a blur effect corresponding to a 
preset in- focus state to a captured image in an image 
processing apparatus, said computer-readable program 
code means comprising: 

first computer-readable program means for 
providing a computer with a function of capturing image 
information including distance information to each 



portion of an object to be photographed; 

second computer-readable program means for 
providing the computer with a function of inputting a 
parameter from which an effective aperture and focal 
length of the assumed image sensing optical system can 
be derived; 

third computer-readable program means for 
providing the computer with a function of designating 
an in-focal pint position of the assumed image sensing 
optical system; 

fourth computer-readable program means for 
providing the computer with a function of calculating 
blur state from the input distance information, the 
designated in-focal pint position, and the input 
parameter ; and 

fifth computer-readable program means for 
providing the computer with a function of applying the 
blur effect to the input image in correspondence with 
the calculated blur state. 

9. An article of manufacture comprising: 

a computer-readable storage medium having 
computer-readable program code means stored to assume 
characteristic of a virtual image sensing optical 
system and apply a blur effect corresponding to an 
in-focus state to a captured image in an image 
processing apparatus, said computer-readable program 
code means comprising: 
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first computer-readable program means for 
providing a computer with a function of capturing image 
information including distance information to each 
portion of an object to be photographed 
5 second computer-readable program means for 

providing the computer with a function of inputting a 
parameter from which an effective aperture and focal 
length of the assumed image sensing optical system can 
be derived; 

10 third computer-readable program means for 

providing the computer with a function of designating 
an in-focal pint position of the assumed image sensing 
optical system; 

fourth computer-readable program means for 

15 providing the computer with a function of calculating a 

confusion circle from the input distance information, 
the designated in-focal pint position, and the input 
parameter; 

fifth computer-readable program means for 
2 0 providing the computer with a function of calculating a 

blur state using a point spread function (PSF) within a 
range corresponding to a size of the calculated 
confusion circle; and 

sixth computer-readable program means for 
25 providing the computer with a function of applying the 

blur effect to the input image in correspondence with 
the calculated blur state. 
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10. An article of manufacture comprising: 
a computer-readable storage medium having 
computer-readable program code means stored to apply a 
blur effect corresponding to an in-focus state to a 
captured image in an image processing apparatus, said 
computer-readable program code means comprising: 

first computer-readable program means for 
providing a computer with a function of capturing image 
information including distance information to each 
portion of an object to be photographed; and 

second computer-readable program means for 
providing the computer with a function of applying the 
blur effect to the input image by overwrite 
sequentially from an image portion having far distance 
information. 

11. An image processing apparatus comprising: 
means for inputting image data including depth 
information; 

means for setting a parameter for expressing a 

lens characteristic ; 

means for setting any one of a focal length, 
F-number f field angle, and effective aperture; 

means for setting a distance for adjusting a 
focus ; 

means for calculating expression j)f an image Xn<& ^ 

texture including a blur from tH& virtual camera v W4lo> c 
setting values and the depth information of the image; 
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and 

means for storing a calculation result in a memory. 

12. An apparatus according to claim 11, further 
comprising means for inputting an image having depth 

5 information in units of pixels of a two-dimensional 

image . 

13. An apparatus according to claim 11, wherein 
a blur function is a point spread function (PSF) 
calculated in units of pixels from the focal length of 

10 a lens, the F-number or aperture, the object distance, 

and the depth information of the image - 

14. An apparatus according to claim 11, wherein a 
blur function has a variable function shape, and is a 
concave function or convex function determined by a 

15 focal pint position and an object distance to be 

calculated . 

15. An apparatus according to claim 11, wherein a 
user can freely change, with a volume, the F-number and 
a focal pint position designated by the user, and a 

20 calculation result using a thumbnail image obtained 

by thinning out an input image is displayed to 
interactively obtain confirmable parameter setting. 

16. An apparatus according to claim 11, wherein a 
user designates a position on a window to determine a 

25 focal pint position using depth information of the 

window . 

17. An apparatus according to claim 11, wherein a 
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user freely changes the F-number with a volume while 
designating a focal pint position on a designated 
window, and a calculation result using a thumbnail 
image obtained by thinning out an input image is 
5 displayed to interactively obtain parameter setting 

capable of confirming designation of the focal pint 
position and the F-number. 

18. An apparatus according to claim 11 , wherein a 
user can change a zoom ratio by changing an f-number. 
10 19. An apparatus according to claim 11, wherein a 

central point of a zoom and a position to get into 
™ focus are set in setting a zoom ratio, and central 

™ coordinates of the zoom and depth information of the 

Q point are used to calculate enlargement on a window and 

O 15 a blur state or to interactively determine a parameter. 

20. An apparatus according to claim 19, wherein an 
outer frame in an area to be enlarged and displayed 
without changing a display ratio is displayed in 
setting the zoom ratio. 
20 21. An apparatus according to claim 11, wherein a 

window is enlarged based on central coordinates of a 
zoom designated by a user, a focal pint position on the 
window is designated to determine the focal pint 
position, the F-number is freely changed with a volume, 
25 and a calculation result using a thumbnail image 

obtained by thinning out an input image is displayed to 
interactively obtain parameter setting capable of 



confirming designation of a central position of the 
field angle, a zoom ratio, and the F-nuraber. 

22. An apparatus according to claim 11, wherein a 
blur state is expressed using influence of coma which 
changes depending on a distance and azimuth from a 
center of an image. 

23. An apparatus according to claim 11, wherein an 
asymmetrical point spread function is calculated for 
coma using a blur function obtained by shifting a 
central position of a radius of a confusion circle in 
accordance with a distance from a central position of 
an image . 

24. An apparatus according to claim 11, wherein 
influence of chromatic aberration is adopted. 

25. An apparatus according to claim 11, wherein a 
range d- A from d on a point spread function for a 
radius (d) of a symmetrical confusion circle is set as 
a chromatic aberration influence range, and red 
aberration or blue aberration is applied. 

26. An apparatus according to claim 11, wherein 
influence of a color shift in air is used. 

27. An apparatus according to claim 11, wherein a 
blue range or red range of a color characteristic at 
not less than a given distance is emphasized or 
decreased. 

28. An apparatus according to claim 11, further 
comprising means for inputting, as a set of units, 



depth information and image information in units of 
pixels or areas for each object. 

29. An apparatus according to claim 11, wherein an 
input image is a set of objects having a predetermined 
distance in units of not pixels but sets of objects. 



